Section: New Results

An adaptive SIR method for block-wise evolving data streams

Participants : Marie Chavent, Jérôme Saracco.

In this work, we consider block-wise evolving data streams. When a semi-parametric regression model involving a common dimension reduction direction $\beta$ is assumed for each block, we propose an adaptive SIR (for sliced inverse regression) estimator of $\beta$. This estimator is faster than usual SIR applied to the union of all the blocks, both from computational complexity and running time points of view. We show the consistency of our estimator at the root-$n$ rate and its asymptotic normality. We also propose an extension of this method to multiple indices model. In simulation studies, we illustrate the good numerical behavior of our estimator. We also provide a graphical tool in order to detect if there exists a drift of the dimension reduction direction or some aberrant blocks of data. We illustrate our approach with various scenarios. We apply this approach on the following real data problem.

As an illustration, we consider a nonlinear inverse problem in remote sensing. The goal is to estimate the physical properties of surface materials on the planet Mars from hyperspectral data. The method is based on the estimation of the functional relationship between some physical parameters $Y$ and observed spectra $X$. For this purpose, a database of synthetic spectra is generated by a physical radiative transfer model. We propose to reduce the high dimension of spectra ($p=352$ wavelengths) with a regularized version of SIR. The need to regularize SIR in very high dimensions is well-known. In practice, the database of synthetic spectra may be so large that it cannot be stored in a computer memory. Thus, a stream of smaller sub-databases is generated and we apply our “SIR datastream” approach to this context.

This work will be submitted for publication very soon and it has been presented in the international conference [31] .